VLP: A Survey on Vision-language Pre-training

Abstract In the past few years, emergence of pre-training models has brought uni-modal fields such as computer vision (CV) and natural language processing (NLP) to a new era. Substantial works have shown that they are beneficial for downstream tasks avoid training model from scratch. So can pre-trained be applied multi-modal tasks? Researchers explored this problem made significant progress. This paper surveys recent advances frontiers in vision-language (VLP), including image-text video-text pre-training. To give readers better overall grasp VLP, we first review its five aspects: feature extraction, architecture, objectives, datasets, tasks. Then, summarize specific VLP detail. Finally, discuss VLP. best our knowledge, is survey focused on We hope shed light future research field.